r 


NPSOR-93-004 


NAVAL  POSTGRADUATE  SCHOOL 

Monterey,  California 


BAYESIAN  PREDICTION  OF  MEAN  SQUARE  ERRORS 
WITH  COVARIATES 


Donald  P.  Gaver 
Patricia  A.  Jacobs 

November  1992 


FedDocs 

D  208.14/2 

NPS-OR-93-004 


Approved  for  public  release;  distribution  is  unlimited. 

Prepared  for: 

Naval  Research  Laboratory-West 
nterey,  CA  93943-5006 


NAVAL  POSTGRADUATE  SCHOOL 
MONTEREY,  CALIFORNIA 


Rear  Admiral  R.  W.  West,  Jr.  Harrison  Shull 

Superintendent  Provost 


This  report  was  prepared  in  conjunction  with  research  funded  by  the 
Naval  Research  Laboratory- West,  Monterey,  California. 

This  report  was  prepared  by: 


UNCLASSIFIED 

SECURI 1 V  CLAs^tHCA  I  ION  OH  HIS  PAGb 


wuuLtY  KNOX  LIBRARY 
NAVAL  POSTGRADUATE  SCHOn. 
MONTEREY  dA  WaJwJS? 


REPORT   DOCUMENTATION   PAGE 


la    RbPOH  I  SECURI  I V  CLASSIHCA I  ION 
UNCLASSIFIED 


lb.    HbSlHICIIVb  MARKINGS 


Form  Approved 
OMB  No.  0704-0188 


>a    SECURITY  CLASSIFICATION  AUTHORITY 


3 — DISTRIBUTION  /AVAILABILITY  OF  REPORT 

Approved  for  public  release;  distribution  is 
unlimited. 


5b.    UbCLAUiilMCA  I  ION/DOWNGRADING  SCHLUULt 


J — PbHI-OHMING  ORGANIC  I  ION  HbPOHl  NUMUbH(!J) 
NPSOR-93-004 


MONI I ORING  ORGANIZATION  RbPOHl  NUMBbR(S) 


1    NAMbObPbHI-OHMINGOHGANIZAIION 

Naval  Postgraduate  School 


6b.  OHHCb  SYMBOL 
(If  applicable) 

OR 


7a     NAMEOhMONIIOHINGOHGANIZAIION 

Naval  Research  Laboratory-West 


5c.    AD  DHbSS  ( City,  State,  and  ZIP  Code) 

Monterey,  CA    93943 


7b.     ADDRESS  ( City,  State,  and  ZIP  Code) 

Monterey,  CA    93943-5006 


5a    NAMb  01-  FUNDING/SPONSORING 
ORGANIZATION 

Naval    Research 


8b.   OFFICE  SYMBOL 
(It applicable) 


■ey 


5 — PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


Laboratory- West 


3c.     AD  DRESS  ( City,  State,  and  ZIP  Code) 

Monterey,  CA    93943-5006 


16.    SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 
ELEMENT  NO. 


PROJECT 
NO. 


1 1 .  TITLE  (Include  Secunty  Classification) 

Bayesian  Prediction  of  Mean  Square  Errors  with  Covariates 


TASK 
NO 


WORK  UNIT 
ACCESSION  NO 


12.    PbRSONALAUTHOR(S) 

Donald  P.  Gaver  and  Patricia  A.  Jacobs 


13a.TYPbOFHbPOHI 

Technical  Report 


13b.  IIMbCOVbHbU 
FROM TO 


1 4.  DA  It  Of-  RLPOR  I  ( Year,  month  day) 

November  1992 


15PAGbCOUNI 
37 


15    SUPPLEMENTARY  NOIAIION 


T5!     SUBJbC I  I  hHMS  (ConOnue  on  reverse  it  necessary  and  identify  by  block  number) 

Hierarchical  Model,  Gaussian  model  with  log-linear  scale 
parameters 


TT 


COSAIICOLibS 


FIELD 


GROUP 


SUB-GROUP 


19.    ABSTRACT  ( Continue  on  reverse  if  necessary  and  identify  by  block  number) 

Estimation  of  mean  square  prediction  error  of  wind  components  is  required  in  the  optimal 
interpolation  (OI)  process  in  numerical  prediction  of  atmospheric  variables.   Previous  work  has 
suggested  that  statistical  models  with  log-linear  scale  parameters  which  include  covariates  can  be 
used  to  predict  mean  square  prediction  errors.   However,  the  parameters  of  the  statistical 
relationships  appear  to  change  over  time.   A  procedure  is  described  to  recursively  update  the 
estimated  parameters.   Data  from  July  of  1991  are  used  to  fit  the  model  parameters  and  to  study 
the  predictive  ability  of  the  recursive  procedure.   This  preliminary  investigation  indicates  that 
observational  and  first  guess  wind  components  can  be  helpful  in  predicting  mean  square 
prediction  error  for  wind  components. 


55.    DISTRIBUTION/AVAILABILITY  OF  ABS !  RAO I 

M  UNCLASSIFIED/UNLIMITED     []  SAME  AS  RPT.        ]  DTIC  USERS 

>2a  NAME  OF  RESPONSIBLE  INDIVIDUAL 

P.  P.  Gaver 

3D  Form  1473,  JUN  86 


21.    ABSTRACT  SECURITY  CLA55ICIATI0N 

UNCLASSIFIED 

22b.  TELEPHONE  (Include  Area  Code) — He" 

(408)  646-2605 


OFFICE  SYMBOL 
OR/Gv 


Previous  editions  are  obsolete. 
S/N    0102-LF-014-6603 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 
UNCLASSIFIED 


BAYESIAN  PREDICTION  OF  MEAN  SQUARE  ERRORS  WITH 

COVARIATES 
by  P.  A.  Jacobs  and  D.  P.  Gaver 

1.     INTRODUCTION  AND  SUMMARY 

Numerical  meteorological  models  are  used  to  assist  in  the  prediction  of 
weather.  Each  run  of  a  numerical  model  produces  forecasts  of  meteorological 
variables  which  are  used  as  preliminary  predictions  of  future  values  of  these 
variables.  These  initial  predictions  are  referred  to  as  first-guess  values.  In 
this  paper  first-guess  values  will  refer  to  the  most  recent  12-hour  forecasts. 

In  certain  areas  of  the  world,  observations  of  forecasted  variables  become 
available.  Prior  to  the  next  run  of  the  numerical  model  a  multivariate 
optimal  interpolation  analysis  updates  a  first-guess  value  of  a  variable  by 
adding  to  it  a  weighted  observed  value  of  the  variable  if  it  is  available.  The 
weight  multiplying  the  observed  value  depends  on  estimates  of  the  mean 
squared  error  of  the  first-guess  value  and  the  mean  squared  error  of  the 
observation;  cf.  Goerss  et  al.,  [1991,  a,  b].  Thus  it  is  of  importance  to  predict 
such  first-guess  squared  errors. 

The  general  problem  of  modeling  and  predicting  mean  square  errors  is 
important  but  not  widely  studied;  see  Davidian  and  Carroll  (1987),  Nelder  and 
Lee  (1992),  Aitken  (1987),  McCullagh  and  Nelder  (1983). 

In  Jacobs  and  Gaver  (1991,  1992)  statistical  models  for  the  error  of  the  first- 
guess  are  used  to  predict  mean  square  error  for  first-guess  wind  components. 
The  models  assume  that  the  error  of  the  first-guess  is  normal  with  mean  0 


and  variance  which  is  a  function  that  is  log-linear  with  suitable  covariates. 
The  cross-validation  results  of  those  papers  suggest  that  covariates  do  have 
some  predictive  ability  for  the  mean  square  errors.  However,  the  relations 
change  over  time. 

In  this  paper  we  introduce  a  procedure  for  recursively  updating  the 
estimated  parameters  of  the  variance  function.  The  approach  is  Bayesian 
with  recursive  updating  using  an  approximation  based  on  the  Laplace 
method;  cf.  deBruijn  (1958). 

In  the  next  section  the  model  is  introduced.  Details  of  the  updating 
procedure  are  also  given. 

The  third  section  presents  results  of  using  the  procedure  to  predict  mean 
square  first-guess  wind  component  errors.  The  data  consist  of  measurement 
and  12  hour  forecasts  (first-guess  values)  of  u  and  v  wind  components  at  850 
mb,  500  mb,  and  250  mb  pressure  levels  from  93  stations  in  North  America, 
25N-75N.  The  measurement  values  (if  available)  are  subtracted  from  the 
first-guess  values  to  obtain  observations  of  the  first  guess  error.  The 
covariates  considered  are  wind  speed  and  resultant  wind,  (the  sum  of  the 
squared  difference  of  the  w~wind  component  at  two  consecutive  12  hour 
periods  and  the  squared  difference  of  the  v-w'md  component  at  the  same 
times).  The  resultant  wind  is  a  measure  of  the  change  in  the  atmosphere. 
Higher  wind  speeds  suggest  more  activity  in  the  atmosphere. 

The  results  of  the  data  analysis  suggest  that  the  covariates  do  have 
predictive  ability.  The  models  using  observed  wind  speed  and  resultant  wind 
have  more  predictive  ability  than  those  using  the  first-guess  values  of  wind 
speed  and  resultant  wind.    Further,  models  that  use  both  wind  speed  and 


resultant  wind  have  more  predictive  ability  than  those  using  either  one  by 
itself.  The  change  of  the  model  parameters  with  time  appears  to  be  slow. 
This  suggests  that  while  the  relationship  of  the  mean  square  error  and  the 
resultant  wind  and  wind  speed  is  changing,  it  may  not  be  necessary  to  update 
the  model  parameters  in  every  period. 

2.     THE  MODEL  AND  STATISTICAL  PROCEDURE 

Let  Y,-(0  denote  the  first-guess  error  at  location  i  at  time  t;  i  =  1,  ...,  L.  Let 
Xj(0  =  (xij(t);  j  =  I,  ...,  p)  denote  the  covariates  at  location  i  at  time  t. 
Consider  the  following  model  for  the  first-guess  errors. 

P{Yt{t)edyt(t);i  =  l...,L\xi(t)^(t),yJ(s),xj(s),s<t,j  =  l...,L}  (21) 

=  n^=exp|-ix((r)(3(r)}exp|-iyi(r)2exp{-xi(r)(3(r)}}^(r) 

where  x;(Op(0  =  X-=i**;M0/'(O/that  is'  given  (MO,  the  first-guess  errors  are 
conditionally  independent  normally  distributed  with  mean  0  and  log  linear 


variances 


o?(0  =  exp{xf(0P(0}  (2-2) 

independent  of  everything  else. 

The  coefficients  (3(0  are  modeled  as  changing  according  to  a  random  walk 

P(f  +  l)  =  P(f )+»('  +  !)  <2-3) 

where  {co(0}  are  independent  multivariate  normal  random  variables  with 
variance-covariance  matrices  [W(t)).  The  matrix  W(t+1)  is  independent  of 
{Yi(s),xi(s),$(s),s<t  i  =  l,...,L} 


In  the  next  subsection  we  suggest  a  Kalman  filter-like  procedure  to 
produce  successive  estimates  of  p(0  as  new  data  become  available.  The 
procedure  is  based  on  a  Laplace  approximation  to  an  integral. 

2.1  An  approximate  Updating  Procedure 

Assume  the  posterior  distribution  of  (3(0  given  [yi(s),  i  -  1,  ...,  L,  s  <t)  is 
multivariate  normal  with  mean  m(t)  and  variance-covariance  matrix  L(0- 
Since  it  is  known  that 

p(r+l)  =  p(r)  +  co(r+l), 

the  prior  distribution  of  p(£+l)  is  multivariate  normal  with  mean  m(0  and 
variance-covariance  matrix 

R(0  =  1(f)  +  W(f+1)  .  (2.4) 

A  description  of  the  procedure  used  to  determine  W(f+1)  appears  in  Section  3. 
The  forecast/ prediction  distribution  of  {Y,(f+1);  i  =  1,  ...,  L}  in  terms  of  data 
up  to  time  t  and  covariate  values  at  time  t  +1  is 

p{Yi{t  +  l)edyi;i  =  l,...,L\Yi(s),xi(u),i  =  l,...,L,s<t,u<t  +  l} 

L  i23) 

=  jn^expj-ixf(t  +  l)bjexp^ 


where 


/p<(+1)(l>)  =  ((2*)"  del  R(t))~  2  exp{-i(b  -  m(f  ))R-'(t)(b  -  m(())'l.        (2.6) 


We  now  approximate  the  integral  by  the  Laplace  method;  cf.  Easton 
(1991),  Cox  and  Hinkley  (1974),  de  Bruijn  (1958).  Let  the  exponent  of  the 
integrand  be 


8(*)  =  -\ 


£(xfb)  +  yf  exp{-X|-b}  +  (b  -  m(f))R-1  (f  )(b  -  m(0)' 


+  K       (2.7) 


where  K  is  a  constant  and  we  let  x,  =  x/(r+l).    Differentiating,  we  obtain 

L 

I 

.i'=i 


j 


£  ^[l "  y«?  exp{-xfb}]  +  2Rj:1(0(b  -  m(0) 


(2.8) 


where  R-.  (0  denotes  the  jth  row  of  R    (t)  and 


a2     rb)__i 

dbjdbkg{  '       2 


Xy2  exp{-xibK'^ + 2Rik 


i=i 


(2.9) 


Use  a  Newton  procedure  to  solve  the  system  of  equations 


°=dfg{h)     j  =  1 P 


(2.10) 


for  m(f+l).    Solve  for  E(t+1)  using  the  second  derivatives  of  g  evaluated  at 
m(f+l);  that  is  Z(t+1)  is  minus  the  inverse  of  the  matrix  whose  (/,  k)  entry  is 


dbjdb^ 


g(m(t  +  l)). 


(2.11) 


The  posterior  distribution  of  p(f+l)  given  {Yi(s),  x,(s),  i  =  1,  ...,  L,  s  =  1,  ...,  f+1}  is 
approximated  by  a  multivariate  normal  distribution  with  mean  m(t+l)  and 
variance  £(f+l).   The  estimate  of  p(f+l)  is  m(f+l). 

The  predicted  mean  square  error  for  the  ith  location  at  time  t+2  is 


exp 


V 


3.     DATA  ANALYSIS 

In  this  subsection  we  report  results  concerning  using  regression-like 
models  for  the  mean  square  error  of  the  first-guess  with  recursively  updated 
parameter  estimates  to  predict  future  mean-square  errors  of  the  first-guess. 

The  data  consist  of  measurement  and  12  hour  forecasts  (first-guess 
values)  of  u  and  v  wind  components  at  the  850  mb,  500  mb  and  250  mb 
pressure  levels  from  93  stations  in  North  America  25N-75N  for  the  month  of 
July  1991.  The  forecasts  are  produced  using  the  NOGAPS  Spectral  Forecast 
Model;  cf.  Hogan  et  al.,  (1991).  Each  station  has  measurement  and  first-guess 
values  for  every  12  hours;  there  are  some  missing  observations  and 
suspicious  values  of  wind  components  equal  to  0.  These  missing  and 
questionable  values  are  deleted  from  the  data  set.  The  measurement  values 
(if  available)  are  subtracted  from  the  first-guess  values  to  obtain  observations 
of  the  error  of  the  first-guess  value. 

Let  U(0;t),  (respectively  V(0;t)),  be  the  observed  w-wind,  (respectively 
p-wind)  component  at  time  t.  Let  U(f;t),  (respectively  V(f;t)),  be  the  first-guess 
w-wind  (respectively  u-wind)  component  at  time  t;  U(f;t)  is  the  forecasted 
value  of  1/(0  made  12  hours  previously.  The  first-guess  error  for  the  w-wind 
component  is 

Y(t)  =  U(f;t)  -  U(0;t).  (3.1) 

The  following  covariates  are  considered  in  the  log-linear  model  for  the  mean 
square  error  of  the  first-guess. 


r(0;0  =  (U(0;0  -  U(0;t- 1))2  +  (V(0;t)  -  V(Q;t-  l)f 


(3.2) 


s(0;f)  =  U(0;t)2 +V{0;t): 


(3.3) 


'(/;')=  (w(/;0-u(/;f-i))2+(v(/;f)-v(/;f-i)): 


(3.4) 


s(/;f)=:  U{f;t)2  +  V(f;t)2 


(3.5) 


The  resultant  wind  r(0;f),  (respectively  r(/;0),  is  a  measure  of  the  observed 

(respectively  forecasted),  change  in  the  wind.   The  variable  s(0;f),  (respectively 

s(/;0),  is  the  observed,  (respectively  forecasted),  wind  speed.    Higher  wind 

speeds  suggest  more  activity  in  the  atmosphere. 

3.1   The  Models 

The  following  models  for  the  mean  square  error  are  considered 

One  Variable  Models:  Observed  Covariates 

1.  Given  /?o(0,  Pitt),  the  first-guess  errors  at  each  location  [Yjtt);  i  =  1,.-.,  L} 
are  independent  normally  distributed  with  mean  0.  The  variance  of 
Y{(t)  is  the  following  function  of  the  observed  resultant  wind  at 
location  i  at  time  t,  r/(0;f) 


Oi(l;f;ri(0;t;)  =  exp{j30(0  +  £l(Orz(0;O} 


(3.6) 


where  rf(0;f)  is  the  observed  resultant  wind  at  location  i  at  time  t  and 
Y{(t)  is  the  first-guess  error  at  location  i  at  time  t. 

Given  (3o(t),  (3\(t),  {Y,-(0;  i  =  1,  •••/  ^}  are  independent  normally 
distributed  with  mean  0.  The  variance  of  Yj(t)  is  the  following 
function  of  the  observed  wind  speed  at  location  i  at  time  t,  sz(0;f) 


ol(l;l;Sl(0;t))  =  exp{j30(0  +  Pl(t)si(0;t)}  (3.7) 

where  s;(0;f)  is  the  observed  wind  speed  at  location  i  at  time  t. 
Two-variable  Model,  Observed  Covariates 

3.  Given  po(t),  p\(t),  feit),  {Yf(0;  i  =  1,  •••,  L)  are  independent  normally 
distributed  with  mean  0.  The  variance  of  Y{(t)  is  the  following  function 
of  both  the  resultant  wind  and  wind  speed  at  location  i  at  time  t 

G2(t;  n(0;t);  Si(0;t))  =  exp{#)(0  +  f3\(t)ri(0;t)  +  (32(Osz(0;0}.  (3.8) 

Similar  one-variable  and  two-variable  models  but  using  first-guess  values  of 
the  covariates  are  also  considered.  In  all  cases  the  first-guess  error  and  the 
covariates  are  all  evaluated  at  the  same  pressure  level. 

The  regression  parameters  (3(0  are  assumed  to  evolve  according  to  the 
random  walk  given  by  (2.3) 
3.2   The  Data  Analysis 

The  results  of  Jacobs  and  Gaver  (1992)  suggest  that  of  the  models  using 
observed  values  for  covariates,  the  models  for  the  850  mb  pressure  level  have 
the  most  predictive  value.  It  is  also  suggested  that  of  the  models  using  first- 
guess  values  for  covariates,  the  models  for  the  250  mb  pressure  level  have  the 
most  predictive  ability.  As  a  result  in  what  follows  we  will  restrict  our 
attention  to  these  two  cases. 

a.     Estimation  and  Prediction  of  850  mb  First-guess  Mean  Square  Errors  using 
Observed  Wind  Covariates. 

The  estimation  procedure  described  in  Section  2  was  used  to  recursively 

estimate  the  regression  parameters  p(0  for  each  of  models  (3.6)  -  (3.8)  for  850 

mb  first-guess  errors  using  observed  850  mb  wind  covariates.   The  initializing 

estimates  of  p(0  are  the  estimates  obtained  using  all  April  data  recorded  in 
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Jacobs  and  Gaver  (1992);  the  initial  variance-covariance  matrix  is  the  identity 
matrix.  The  estimates  from  April  are  used  since  April  appeared  to  have  more 
predictive  ability  for  July  than  February,  cf.  Jacobs  and  Gaver  (1992). 

The  variance-covariance  matrix  of  the  innovation  Wt  of  the  random 
walk  (2.3)  was  taken  to  be  a  constant  times  the  identity  matrix.  Preliminary 
explorations  based  on  values  of  the  predictive  log-likelihood  using  different 
values  of  the  constant  suggest  that  for  purposes  of  prediction,  the  constant 
should  be  very  small.   In  what  follows  we  set  the  constant  equal  to  0. 

Figure  1,  (respectively  Figure  2)  presents  plots  of  the  estimates  of  the 
slopes,  e.g.,  pi(t)  and  fait),  as  a  function  of  time  which  is  labeled  1,  2,  ....  for  u- 
wind  (respectively  p-wind)  component  error. 

Figure  1  presents  the  values  of  estimates  of  the  parameter  multiplying 
r{(0;t)  (respectively  S{(0;t))  for  the  one  parameter  models  (3.6)  and  (3.7)  in  the 
upper  graph.  The  lower  graph  presents  the  values  of  the  estimates  of  the 
parameter  multiplying  ri(0;t),  (respectively  S{(0;t))  for  the  two-variate  model. 
There  appears  to  be  a  slight  trend  in  the  estimates. 

Figure  2  presents  the  values  of  the  estimate  of  the  parameters  multiplying 
r{(0;t),  (respectively  si(0;t))  as  o,  (respectively  +),  for  the  respective  one-variate 
models.  The  values  of  the  estimates  of  the  parameters  multiplying  r{(0;t), 
(respectively  S{(0;t))  for  the  two-variate  model  are  presented  as  x  (respectively 
V)  for  each  time  t.  These  graphs  suggest  more  evidence  of  a  trend  in  the 
estimates.  Note  that  the  estimates  of  the  slopes  are  positive.  Hence  increased 
values  of  the  resultant  wind  r{(0;t)  and/or  wind  speed  Si(0;t)  are  associated 
with  increased  variance  of  the  first-guess  value.    This  is  plausible  physically, 


since  a  large  value  of  rj(0;t)  is  indicative  of  a  change  in  the  atmosphere  and  a 
large  value  of  Sj(0;t)  is  indicative  of  greater  activity  in  the  atmosphere. 

To  assess  the  predictive  ability  of  the  models,  the  models  with  parameters 
estimated  at  time  t  are  used  to  forecast  the  variances  of  the  first-guess  errors  at 
time  M-l. 

One  procedure  to  informally  assess  the  predictive  ability  of  the  models  is 
by  binning  the  data.  To  assess  models  (3.6)  and  (3.8)  the  data  (y,(0,  r;(0;0, 
Sj(0;O)  are  binned  into  10  bins  based  on  ordering  the  values  of  r,(0;0  for  all 
time  t  from  smallest  to  largest.  The  data  in  the  first  bin  correspond  to  the 
smallest  values  of  rf(0;0;  the  data  in  the  10th  bin  correspond  to  the  largest 
values  of  r,-(0;f).  Each  bin  contains  about  l/10th  of  the  data  with  the  10th  bin 
containing  a  few  more  data.  The  averages  of  the  predictive  variances  for 
models  (3.6)  and  (3.8)  are  computed  for  each  bin.  The  average  yi(t)2  is  also 
computed  for  each  bin. 

To  assess  models  (3.7)  and  (3.8)  the  same  procedure  is  used  but  the 
binning  is  based  on  the  values  of  Sj(0;t). 

Figures  3  and  4  present  graphs  of  the  log[average  x/i(t)2]  in  each  bin  versus 
log  [average  predictive  variance]  in  each  bin  for  models  (3.6)  and  (3.8)  and 
models  (3.7)  and  (3.8).  If  a  model  were  perfect,  the  points  should  be  close  to 
the  45°  line  shown. 

The  figures  suggest  that  of  the  two  one-variate  models,  the  one  using  the 
resultant  wind  ri(0;t)  has  the  better  predictive  ability.  The  two-variate  model 
appears  to  have  similar  predictive  ability  to  the  model  using  only  r{(0;t). 

Another  procedure  to  assess  predictive  ability  is  to  compute  the  log- 
likelihood  using  estimated  values  of  (3(0  (eg.  m(0)  in  the  term  for  the  first 
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guess  errors  at  time  t+1.  Larger  values  of  the  (predictive)  log-likelihood 
indicate  better  predictive  ability.  The  (predictive)  log-likelihood  up  to 
addition  and  multiplication  by  constants  is 

(3.9) 


T-\ 


'=-1 


f=] 


Xx.U  +  lMO  +  X^^  +  Dexpl-x^r  +  lMO} 


«=i  <=i 


Table  1  presents  values  of  /  for  the  one-time  step  ahead  predictions  of  the 
variance;  these  values  appear  in  the  column  Iterative.  Also  displayed  are  the 
values  of  /  obtained  by  estimating  the  parameters  once  using  all  the  data;  this 
value  of  /  is  a  goodness-of-fit  value  and  appears  in  the  column  labeled  All; 
the  estimates  used  to  obtain  the  goodness-of-fit  value  of  /  are  those  appearing 
for  July  in  Table  7  of  Jacobs  and  Gaver  (1992). 

Four  models  are  considered:  constant  variance  (no  dependence  on 
variables),  two  one-variate  models  (3.6)  and  (3.7)  and  the  two-variate  model 
(3.8).  The  parameter  of  the  constant  variance  model  using  all  the  data  is 
estimated  using  maximum  likelihood;  this  estimate  is  used  to  calculate  the 
goodness-of-fit  value  of  /  for  the  constant  variance  model. 

The  goodness-of-fit  constant  variance  value  of  /  is  smaller  than  the 
prediction  values  of  /  using  models  with  covariates.  This  behavior  indicates 
that  the  covariates  do  have  predictive  ability.  The  prediction  value  of  /  for 
the  two-variate  prediction  model  is  larger  than  that  for  either  one-variate 
model  indicating  that  both  covariates  have  some  predictive  ability.  Of  the 
two  one-variate  models,  the  one  using  rj(0;t)  has  the  larger  prediction  value 
of  /.  The  closeness  of  the  prediction  values  of  /  obtained  by  iteratively 
updating  the  estimates  and  using  them  to  predict  variance  of  the  next  time 
period   and   the  goodness-of-fit  values   of   /    obtained    by   using   model 
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parameters  estimated  from  all  the  data  suggest  that  the  updating  procedure  is 
doing  very  well.  Note  that  for  the  one-variate  model  using  rtfOjt),  the 
prediction  values  of  /  are  larger  than  the  goodness-of-fit  values  of  /;  this 
suggests  that  there  is  systematic  change  in  the  values  of  p  over  time  for  this 
model. 


TABLE  1 
VALUE  OF  LOG-LIKELIHOOD  OBSERVED  COVARIATES,  850  mb 


Data 

Model 

Iterative 

All 

(Prediction) 

(Goodness-of-Fit) 

w-wind 

Constant   Variance 
One-variate 

-9610.7 

-9510.0 

r(0;0 

-8544.1 

-9098.3 

s(0;0 

-8607.9 

-8596.2 

Two-variate 

-8489.2 

-8467.5 

i>-wind 

Constant   Variance 
One-variate 

-9692.7 

-9620.2 

r(0;t) 

-8730.9 

-9294.1 

s(0;t) 

-8769.2 

-8756.5 

Two-variate 

-8663.8 

-8652.4 

b.     Estimation  and  Prediction  of  First-guess  Mean  Square  Errors  using  First- 
guess  Wind  Covariates 

The  recursive  estimation  procedure  in  Section  2  was  used  to  estimate  the 

regression  parameters  p(0  for  each  of  models  (3.6)  -  (3.8)  for  250  mb  first-guess 

errors  using  first-guess  250  mb  wind  covariates;  that  is,  the  first-guess  wind 

speed  at  location  i  at  time  t  at  the  250  mb  level,  Sj(f;t)  replaces  Sj(0;t),  etc.   The 

initializing  estimates  of  p(f)  are  the  estimates  obtained  using  all  April  data 

recorded  in  Jacobs  and  Gaver  (1992);  the  initial  variance-covariance  matrix  is 

the  identity  matrix.    The  estimates  from  April  are  used  since  April  appears  to 

have  somewhat  more  predictive  ability  than  February  for  July;  Jacobs  and 

Gaver  (1992). 
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Once  again,  preliminary  exploratory  work  using  the  resulting  value  of  the 
predictive  log-likelihood  indicates  that  setting  the  variance-covariance  matrix 
of  the  innovation  of  the  random  walk  equal  to  0  gives  the  best  predictions. 
This  suggests  that  the  change  in  the  relationship  of  the  mean  square  error  and 
the  covariates  is  slow. 

Figure  4,  (respectively  Figure  5)  presents  plots  of  the  values  of  the 
estimates  for  the  250  mb  w-wind  component  errors  and  the  250  mb  z^-wind 
component  errors.  The  values  of  the  estimates  multiplying  r(f;t), 
(respectively  s(/;0)  are  represented  by  o,  (respectively  +)  for  the  one-variate 
models.  For  the  two-variate  model,  the  estimates  multiplying  r(f;t), 
(respectively  s(f;t))  are  presented  as  x,  (respectively  V).  The  figures  suggest 
evidence  of  a  trend  in  the  estimates.  Note  that  once  again  all  the  estimates 
are  positive.  Thus,  increased  first-guess  resultant  wind  and/or  wind  speed 
tends  to  increase  the  mean  square  error 

To  assess  the  predictive  ability  of  the  models,  the  models  with  parameters 
estimated  at  time  t  are  used  to  forecast  the  variances  of  the  first-guess  errors  at 
time  t+1. 

One  procedure  to  informally  assess  the  predictive  ability  of  the  models  is 
by  binning  the  data.  The  data  are  binned  as  in  the  previous  subsection. 
Figures  7  and  8  present  graphs  of  the  log  [average  yj(t)2]  in  each  bin  versus  log 
[average  predicted  variance]  in  each  bin  for  models  (3.6)  and  (3.8)  and  models 
(3.7)  and  (3.8).    If  a  model  were  perfect,  the  points  should  be  close  to  the  45° 

line  shown. 

Table  2  presents  values  of  /,  given  by  (3.9),  for  the  one-time  step  ahead 
prediction  of  variance;  these  values  appear  in  the  column  iterative.     Also 
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displayed  are  the  values  of  /  obtained  by  estimating  the  model  parameters 
once  using  all  the  data;  this  value  of  /  is  a  measure  of  goodness-of-fit  and 
appears  in  the  column  labeled  All;  the  estimates  used  in  regressions  with 
covariates  for  the  goodness-of-fit  evaluation  of  /  are  those  appearing  for  July 
in  Table  11  of  Jacobs  and  Gaver  (1992).  The  constant  variance  estimate  using 
all  the  data  is  the  maximum  likelihood  estimate.  The  results  for  four  models 
are  presented;  constant  variance  models,  one-variate  models  (3.6)  and  (3.7), 
and  the  two-variate  model  (3.8). 

Note  that  all  of  the  iterative  prediction  values  of  /  for  the  models  with 
covariates  are  larger  than  those  for  the  constant  variance  goodness-of-fit 
value;  this  suggests  that  the  covariates  have  some  predictive  value. 

The  iterative  prediction  value  of  /  for  the  one-variate  model  using  r(f;t)  is 
greater  than  the  goodness-of-fit  value  obtained  by  using  parameters  estimated 
using  all  the  data;  this  suggests  that  there  is  a  systematic  change  in  the  model 
parameter  values  over  time.  For  the  other  regressions  using  covariates  the 
prediction  values  of  /  are  smaller  than  their  corresponding  goodness-of-fit 
values  but  not  by  much.  The  model  that  maximizes  the  prediction  values  of 
/  is  the  two-variate  model  suggesting  that  both  covariates  have  some 
predictive  value.. 

Figures  9  and  10  and  Table  3  present  results  for  models  of  the  variance  of 
the  first-guess  error  at  500  mb  level  using  500  mb  first-guess  covariates.  The 
implications  of  the  results  are  similar  to  those  of  the  250  mb  results.  The 
predictive  ability  of  the  recursively  estimated  two-variate  model  appears 
somewhat  greater  at  the  250  mb  level  than  the  500  mb  level;  this  conclusion  is 
based  on  the  values  of  (1 2  -  lc)/^c  where  1 2  is  the  prediction  value  of  /  for  the 
two-variate  model  and  lc  is  the  goodness-of-fit  value  of  /  for  the  constant 
variance  model;  the  value  of  this  fraction  is  larger  for  250  mb  than  for  500  mb. 
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TABLE  2 
VALUE  OF  LOG-LIKELIHOOD  FIRST-GUESS  COVARIATES,  250  mb 


Data 

Model 

Iterative 

All 

(Prediction) 

(Goodness-of-Fit) 

u-wind 

Constant   Variance 
One-variate 

-14,589.4 

-14,516.5 

r(f;t) 

-13,640.6 

-14,474.7 

slfit) 

-13,598.3 

-13,575.5 

Two-variate 

-13,599.5 

-13,579.4 

v-wind 

Constant   Variance 
One-variate 

-14,429.1 

-14,363.1 

r(/;0 

-13,389.2 

-14,270.1 

s(f;t) 

-13,388.4 

-13,363.8 

Two-variate 

-13,351.1 

-13,329.6 

TABLE  3 
VALUE  OF  LOG-LIKELIHOOD  FIRST-GUESS  COVARIATES,  500  mb 


Data 

Model 

Iterative 

All 

(Prediction) 

(Goodness-of-Fit) 

u-wind 

Constant    Variance 
One-variate 

-11,237.3 

-11,213.5 

rtfit) 

-10,561.2 

-11,176.8 

s(f;t) 

-10,518.7 

-10,499.6 

Two-variate 

-10,520.0 

-10,489.0 

i>-wind 

Constant   Variance 
One-variate 

-11,204.0 

-11,160.9 

rtfit) 

-10,431.0 

-11,118.2 

s(/;0 

-10,423.0 

-10,411.9 

Two-variate 

-10,411.8 

-10,393.3 
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